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METHOD AND APPARATUS FOR 
MODIFYING SCREEN RESOLUTION BASED 
ON AVAILABLE MEMORY 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention pertains to video processing. More 
particularly, this invention relates to modifying screen reso- 
lutions. 

2. Background 10 
The display of video images has become extremely 

important in the modern world, including typical computer 
uses, such as multimedia games and video conferencing, as 
well as other entertainment uses, such as movies. Modern 
computers, as well as other commercially available devices, 15 
typically include the ability to display video images, many 
of which allow display of real-time motion video. Examples 
of such systems include display devices designed for use 
with typical computer systems. 

One problem encountered in many systems is the amount 20 
of memory required to display video images. Mass storage 
devices, such as CD-ROMs, digital video disks (DVDs) and 
hard disk drives are relatively inexpensive. However, the 
process of displaying video images typically requires trans- 
fer of these images from the mass storage device (or other 25 
source) to a more expensive random access memory (RAM), 
typically referred to as a frame buffer. The RAM used as the 
frame buffer is often video RAM (VRAM), which is par- 
ticularly suited to displaying video, but which is also very 
expensive. The amount of VRAM within a system can often 30 
noticeably affect the overall system price. Thus, it would be 
beneficial to provide a way to reduce the amount of VRAM 
necessary for video images. 

Additionally, modern computer systems typically provide 
users with the ability to alter screen resolution and color 35 
resolution for their display devices. The screen resolution 
refers to the number of pixels displayed by the display 
device, while the color resolution refers to the number of 
different colors which each pixel can have. Larger screen 
and color resolutions provide for more realistic video 40 
images, but also require additional memory, including 
VRAM, for storage and display. Thus, it would be beneficial 
to provide a way to reduce the amount of memory used for 
video images without significantly reducing the image qual- 
ity. ' 45 

Thus, a need exists for a method and apparatus for 
modifying screen resolution based on available memory. 

SUMMARY OF THE INVENTION 

A method and apparatus for modifying screen resolution 50 
based on available memory is described herein. According 
to the present invention, a first screen resolution for a display 
is first identified. Then, a second screen resolution for at 
least a portion of the display is created, wherein the creating 
is based on the first screen resolution and an amount of 55 
memory to be used for the at least a portion of the display. 

According to one embodiment, the first screen resolution 
corresponds to a video signal received from a computer 
system while the second screen resolution corresponds to a 
motion video signal received from a remote video confer- 60 
encing camera. In this embodiment, the second screen 
resolution is determined based on the amount of frame buffer 
memory available for the display of the motion video signal. 
Also, the motion video signal is overlaid on the video signal 
received from the computer system, allowing both images to 65 
be displayed concurrently despite the differing screen reso- 
lutions. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example 
and not limitation in the figures of the accompanying 
drawings, in which like references indicate similar elements 
and in which: 

FIG. 1 illustrates a display of a video multiplexing system 
according to one embodiment of the present invention; 

FIG. 2 illustrates a video multiplexing system according 
to one embodiment of the present invention; 

FIGS. 3, 4, and 5 are a more detailed illustration of the a 
video multiplexing system of FIG. 2 according to one 
embodiment of the present invention; 

FIG. 6 illustrates a video format analyzer according to one 
embodiment of the present invention; 

FIG. 7 is a flowchart illustrating the steps associated with 
establishing a video locked state between two signals 
according to one embodiment of the present invention; 

FIG. 8 illustrates a storage medium including one 
embodiment of the present invention; 

FIG. 9 is a flowchart illustrating the steps followed in 
modifying a horizontal screen resolution according to one 
embodiment of the present invention; 

FIG. 10 illustrates multiple scan lines of a screen accord- 
ing to one embodiment of the present invention; 

FIG. 11 is a liming diagram illustrating pixel clocks 
according to one embodiment of the present invention; 

FIG. 12 is a flowchart illustrating the steps followed in 
identifying a proper horizontal resolution according to one 
embodiment of the present invention; and 

FIG. 13 illustrates a display according to an alternate 
embodiment of the present invention. 

DETAILED DESCRIPTION 

In the following detailed description numerous specific 
details are set forth in order to provide a thorough under- 
standing of the present invention. However, it will be 
understood by those skilled in the art that the present 
invention may be practiced without these specific details. In 
other instances well known methods, procedures, 
components, and circuits have not been described in detail 
so as not to obscure the present invention. 

Some portions of the detailed descriptions which follow 
are presented in terms of algorithms and symbolic repre- 
sentations of operations on data bits within a computer 
memory. These algorithmic descriptions and representations 
are the means used by those skilled in the data processing 
arts to most effectively convey the substance of their work 
to others skilled in the art. An algorithm is here, and 
generally, conceived to be a self-consistent sequence of steps 
leading to a desired result. The steps are those requiring 
physical manipulations of physical quantities. Usually, 
though not necessarily, these quantities take the form of 
electrical or magnetic signals capable of being stored, 
transferred, combined, compared, and otherwise manipu- 
lated. It has proven convenient at times, principally for 
reasons of common usage, to refer to these signals as bits, 
values, elements, symbols, characters, terms, numbers, or 
the like. It should be borne in mind, however, that all of these 
and similar terms are to be associated with the appropriate 
physical quantities and are merely convenient labels applied 
to these quantities. Unless specifically stated otherwise as 
apparent from the following discussions, it is appreciated 
that throughout the present invention, discussions utilizing 
terms such as "processing" or "computing" or "calculating" 
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or "determining** or "disp laying" or the like, refer to the 
action and processes of a computer system, or similar 
electronic computing device, that manipulates and trans- 
forms data represented as physical (electronic) quantities 
within the computer system's registers and memories into 5 
other data similarly represented as physical quantities within 
the computer system memories or registers or other such 
information storage, transmission or display devices. 
Definitions 

"Color space" is used herein to mean a representation of to 
a set of colors. Typical color spaces include RGB, YIQ, 
YUV and YCrCb. 

"Computer video" and "computer video signal" are used 
herein to mean an analog RGB video signal or signals 
generated by any computer or similar hardware device. 15 
Further, such term as used herein includes both "video 
content** as well as format and timing information that is 
separate from the "video content". 

"Far side image" is used herein to mean the motion video 
image sent from a remote site or sites participating in a video 20 
conferencing session. 

"Group clock" or "GCLK" is used herein to mean a 
derivative of the "synthesized pixel clock". According to 
one embodiment, the GCLK is the "pixel clock" divided by 
four. 25 

"Horizontal total" or "H rofa/ " is used herein to mean the 
number of pixel groups contained in a single horizontal scan 
line. H total is the sum, as measured in pixel groups, of the 
horizontal pulse sync width, the horizontal back porch 
("H ") the horizontal active time, and the horizontal 30 
fxonTporch ("H^, p^/,")- 

"Motion video signal" is used in the illustrated embodi- 
ment to mean the "video signal" originating in the video 
conferencing system outside of the computer via a video 
camera, VCR, television, etc. However, as discussed in more 35 
detail below, the zooming of the present invention can use 
any video signal. 

"Multiplexing" is used herein to mean the act of combin- 
ing separate sub-images each of which may have differing 
"video envelopes". 40 

"Near side image" is used herein to mean the motion 
video image generated by a local camera in a video confer- 
encing environment. It is optionally displayed on the local 
screen. 

"Pixel clock" is used herein to mean the periodic digital 45 
clock associated with the "computer video signal" or other 
"video signal" from which all components of the "video 
signal" are referenced. It is preferably a high frequency 
signal, the rate of which defines when the pixels are updated 
as well as how many pixels are on a horizontal scan line. A 50 
derivative of the "pixel clock" known as "group clock" is 
used to create the horizontal sync component of the "video 
envelope" of the computer video signal. "Original pixel 
clock" is used herein to mean the pixel clock of the source 
of the "video signal", while "synthesized pixel clock" is 55 
used herein to mean that pixel clock generated by a phase- 
locked loop which is iteratively adjusted so that it tracks, but 
is not necessarily equal to, the "original pixel clock**. 

"Pixel group" is used herein to mean four pixels, although 
it will be understood that in other embodiments, a different 60 
number may instead be used. 

"Video content" or "video information*' is used herein to 
mean a temporal and spatial dependent stream of data 
representing, in the illustrated embodiment, Red, Green and 
Blue components of an image, the latter of which may be 65 
either static or dynamic. "Video content*' may be separated 
from the "video envelope" in the "video signal** for the 
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purpose of compression, encoding, or other image process- 
ing purposes. For example, "video content" could be sepa- 
rated from its "video envelope" and multiplexed with 
another "video signal" for a picture-in -picture (PIP) effect. 

"Video envelope" is used herein to mean a composition of 
signal components with specific timing relationships used to 
convey a video image between two points. Signal compo- 
nents typically include horizontal sync and vertical sync. 
"Video content" is inserted into the "video envelope" to 
result in a complete "video signal". If no "video content" is 
inserted into the "video envelope", the resulting "video 
signal" appears as a solid black image when viewed on a 
display device. 

"Video signal" is used herein to mean a composite signal 
containing "video content" within a "video envelope". 
Overview of the Video Multiplexing System 

FIG. 1 illustrates a display of a video multiplexing sys- 
tem. A display 100 is illustrated which includes windows 
102, 104, and 106. Such windows may be generated by a 
video conferencing application, such as the local and remote 
windows, or by any of a wide variety of other applications, 
such as a game or other application. It should be noted that 
virtually any number of windows can be displayed on the 
display 100, and that a window could be enlarged to be full 
screen (that is, encompass the entire display). 

The display 100 has an associated screen and color 
resolution. The screen resolution refers to the number of 
pixels which are displayed on the screen and the color 
resolution refers to the number of different colors which 
each pixel can be. Conventional display devices typically 
support a wide range of screen and color resolutions. 
Although a large number of screen resolutions exist, typical 
screen resolutions include 640x480 (307,200 pixels), 800x 
600 (480,000 pixels), 1024x768 (786,432 pixels), 1280x 
1024 (1,310,720 pixels), and 1600x1200 (1,920,000 pixels). 
Similarly, although a large number of color resolutions exist, 
typical color resolutions include 8-bit (256 colors), 16-bit 
(65,536 colors), 24-bit (16,777,216 colors), and 32-bit 
(4,294,967,296 colors). 

The amount of storage space required for an image can be 
calculated by multiplying the number of pixels by the color 
resolution. By way of example, 1024x768 screen resolution 
and 16-bit color resolution requires 1,572,864 bytes, which 
equals 1.5 Megabytes (MB). 

FIG. 2 illustrates a video multiplexing system according 
to one embodiment of the present invention. A computer 210 
as illustrated can be any of a wide variety of general purpose 
computers, such as a desktop PC, laptop PC, or a network 
PC-like device. Computer 210 provides an RGB analog 
computer video signal 212 (including image, timing and 
format information) to the multiplexing system 213 of the 
present invention. An additional data and control path 215 is 
established, such as by coupling the computer's parallel port 
to the video multiplexing system 213. This data and control 
path 215 is used to send messages to and from the computer 
210 and the video multiplexing system 213. Alternatively, 
other uni-directional and/or bi-directional ports, such as a 
serial port, Universal Serial Bus (USB) port, other IEEE 
1394-compliant ports, etc., may be used for the data and 
control path 215. 

The video multiplexing system 213 combines the com- 
puter video signal 212 generated from the computer 210 
with incoming video signals from a local camera 222 and 
from a remote camera (not shown), and provides the result- 
ing output on a monitor 220. A composite image which 
combines the images from the computer 210, the video from 
the remote camera, and the video from the local camera 222 
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is then output from the video multiplexing system 213 for 
display on the monitor 220, and the image from the local 
camera 222 is sent to a video conferencing encoder (part of 
system 213) where it is appropriately encoded before trans- 
mission to the remote system. 5 
Configuration and Operation of the Video Multiplexing 
System 

FIGS. 3, 4, and 5 illustrate a video multiplexing system 
according to one embodiment of the present invention. 
During operation, the computer video signal 212 can pass 10 
directly through the video multiplexing system 213 to the 
video amplifier 316 and the monitor 220. Thus, the video 
multiplexing system passes the computer video signal 212 
without modification. Alternatively, the computer video sig- 
nal 212 may be combined, in real-time, with video generated 15 
from a second video source(s). According to one 
embodiment, wherein the multiplexing system of the present 
invention is used in a video conferencing environment, the 
second video source(s) is a local camera 222 and/or a remote 
camera. Thus, the motion video signals generated from the 20 
local and/or remote cameras should operate within the same 
timing and format as the computer video graphics generated 
by the computer 210. Accordingly, in one embodiment the 
incoming motion video signals are converted to analog RGB 
signals in order to match the color space of the computer 25 
video and the monitor 220, and the timing parameters of the 
computer video signal 212, including horizontal and vertical 
sync and active video time, are analyzed in order to syn- 
chronize the motion video signals with the computer video 
signal 212. The multiplexing system 213 accomplishes this 30 
in a manner described in more detail below. 
I. Motion Video processing 

According to one embodiment of the present invention, 
the motion video is provided from a local camera 222 
(near-side video images) and a remote video camera (far- 35 
side video images), both of which operate according to 
defined NTSC, PAL or other such standards for video, 
including receipt of compressed video via a telephone 
interface 431. In order to combine the motion video directly 
with the video signal 212 from the computer 210, the video 40 
information contained in the NTSC/PAL-formatted or com- 
pressed motion video signal is converted to an analog RGB 
format. Thus, the motion video, including video images and 
other information, generated by both the local video source 
and the remote video source is captured and converted by 45 
video converter 335a or 3352> to a 4:2:2 YUV formatted data 
stream. If the motion video is either NTSC or PAL 
formatted, it is provided to an NTSC/PAL decoder 335a. 
However, if compressed video is received via a telephone 
interface 431 then it is instead decoded in an H.320 decoder 50 
3356. The conversion process of decoders 335a and 3356 is 
well known to those skilled in the art, and thus will not be 
discussed further except as it pertains to the present inven- 
tion. 

The formatted data is output from decoders 335a and 55 
3356 and placed into a VRAM frame buffer 339 by the video 
co-processor 337. VRAM 339 is preferably a 2 MB VRAM. 
The video co-processor 337 converts and scales the YUV 
data to digital RGB by reading out the data from the VRAM 
339, processing the data to convert same to digital RGB 60 
data, and writing the thus digitized RGB data back into the 
VRAM 339. The video co-processor 337 scales the YUV 
data in its received format, a 352x288 resolution image in 
the illustrated video conferencing system, to the appropriate 
RGB screen resolution (e.g., 1280x1024) and to the window 65 
size in which the motion video is to be displayed through a 
process well-known to those skilled in the art. Thus, the 
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co-processor 337 is also referred to as a "scaler" or "scaling 
engine". The co-processor 337 includes multiple user- 
accessible registers (not shown) which can be set with the 
appropriate window size. The co-processor 337 calculates, 
using interpolation, additional pixel values for the image 
and/or eliminates pixels from the image in accordance with 
the window size. According to one embodiment, the video 
co-processor 337 is a Weitck Video Power co-processor 
Model No. 9130-PFD. However, in alternate embodiments 
similar co-processors as known to those skilled in the art are 
substituted. 

The digital information is then converted to an analog 
RGB signal by a video digital to analog (D/A) converter 341 
before it is forwarded to the video switch 314. As illustrated 
in FIG. 3, the entire foregoing process is accomplished 
under the control of a system controller 332 through bus 
arbitration and control logic 345. According to one 
embodiment, the controller 332 is a digital signal processor 
(DSP) Model No. 320C80 available from Texas Instruments 
of Dallas, Tex. Similarly, the video D/A converter 341 is a 
D/A converter Model No. TVP3026, also available from 
Texas Instruments. However, in alternate embodiments 
similar DSPs and/or D/A converters as known to those 
skilled in the art are substituted. 

Once the motion video signal(s) is thus converted, it is 
now ready to be directly combined in the switch 314 with the 
computer video signal 212 from the computer 210. 
However, before the motion video signals (local and remote) 
and computer video signal 212 are combined, the signals are 
synchronized. Synchronization of these signals will now be 
discussed. 

II. Synchronization of the Motion Video and Computer 
Video Signals 

As discussed above, the present invention can be adapted 
for use with almost any computer system, including a variety 
of display types/formats (e.g., any of the standard VESA 
video formats). In effecting synchronization (in format and 
timing) of the motion video signal and the computer video 
signal, the format of the incoming computer video signal is 
initially unknown, and thus the vertical and horizontal 
parameters of such signal are unknown. As set forth above, 
such horizontal and vertical parameters are determined in 
order to synchronize the motion video signal and the com- 
puter video signal. 

In order to synchronize the motion video signal with a 
computer video signal of an unknown format, it is important 
that the signals' vertical sync (V J>1M J and horizontal sync 
(Hsync) signals, as well as active video time, occur coinci- 
dentally. It is further desired to generate a pixel clock for the 
motion video signal which tracks or is based on the original 
pixel clock of the computer video signal 212, and to use this 
generated pixel clock as the basis to create synchronization. 
This generated pixel clock may or may not be substantially 
identical to the pixel clock of the computer video signal 212, 
based on the "zoom factor'*, as discussed in more detail 
below. The video format analyzer and synchronizer (VFAS) 
324 is used to effect these tasks. 

A. Video Format Analyzer and Synchronizer 

As seen with reference to FIGS. 3 and 5, portions of the 
computer video signal 212 (including vertical sync (V^), 
horizontal sync (H^), and the blue component of the 
computer video signal 212) are provided to the video format 
analyzer and synchronizer (VFAS) 324, which analyzes the 
computer video signal 212 to extract video timing informa- 
tion. Basically, the VFAS 324 duplicates the envelope of the 
computer video signal 212 from computer 210, so that the 
signals* envelopes are synchronized. In the illustrated 
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embodiment, WAS 324 includes a programmable frame components of the computer video signal 212 will herein- 

controller (FC) 311, a video format analyzer (VFA) 318 and after be referred to as "external H^^." and "external V^", 

a wide -band phase locked loop (PLL) 319, although the respectively Accurate determination of H ttMaI and V ww/ are 

frame controller 311, VFA 318 and PLL 319 may each thus important in the synchronization of the motion video 

constitute separate devices. A more detailed description of 5 and computer video signals and will be discussed in more 

the operation of the VFAS 324 and its components is detail below, 

discussed below. 2. Phase- Locked Loop (PUS) 

1. Programable Frame Controller (FC) In addition to the FC 311, the VFAS 324 also includes 
VFAS 324 includes a programmable frame controller 311. PLL 319. While some components of the video envelope of 
The FC 311 includes a series of programmable counters (not 10 the computer video signal 212 may be simply extracted, 
shown) and performs typical video frame controller tasks as such as and V^^ the original pixel clock of the 
are well known in the art, such as the generation of and computer video signal 212 is not contained therein and is 
V^,^, and the generation of sequential frame buffer memory synthesized from the computer video signal 212. The pur- 
addresses for display purposes. Therefore, the operation of pose of the PLL 319 is to create (or generate) this pixel clock 
FC 311 will not be discussed in further detail except as it 1 5 and to maintain synchronization of the FC 311 with the 
pertains to the present invention. computer video signal 212. 

FC 311 is used in determining the timing parameters (that According to one embodiment, this is accomplished 
is, the video envelope) of the output RGB analog video through the application of the external to the reference 
signal 322 which will be provided to the monitor 220. All input of the PLL 319 and the internal H^^. from the output 
timing parameters are thus referenced to the clock signal of 20 of FC 311 to the feedback input of the PLL 319. It will thus 
the FC 311. As the frequency of the synthesized pixel clock be appreciated that PLL 319 attempts to maintain the feed- 
frequently exceeds the frequency capability of many com- back frequency (that is, the frequency of internal H^) at 
mon logic devices, the synthesized pixel clock is divided to the same frequency as the reference frequency (that is, the 
a lower frequency signal. This signal is then used as the FC's frequency of external H sync ). In one implementation, the 
clock. In one embodiment, the FC's clock is the synthesized 25 PLL 319 is preferably an ICS 1522 chip available from 
pixel clock (generated by the PLL 319— discussed below) Integrated Circuit Systems, Inc. of Valley Forge, Pa. 
divided by four, which will hereinafter be referred to as the Upon achieving a "synchronized" state, the PLL 319 
Group Clock (GCLK). According to one implementation, output (the synthesized pixel clock) will operate at a fre- 
divider logic available in the video D/A converter 341 is quency as determined according to the following equation: 
used to divide the synthesized pixel clock. However, it is to 30 

be appreciated by those skilled in the art that any similar ^ ^^-^H^/x^,^/ //Jrnc xZoom Factor 

logic capable of scaling frequency may be used. ^though GiventhatthcGC LK signal is Hthe frequency of that of the 

division is shown in FIG. 5 as occurring outside of the video . . . . , , T~ , .. l "T ••. j 

D/A converter, it is to be understood that it has been synthesized p«el clock, its frequency may thus be described 

illustrated as such for simplicity and ease of understanding 35 45 0 ows * 

of the present invention, and not meant as a limitation. f m dH vf v7 . nm Rflrtnr 

J\ . , . ,„ , „ , . \}CLK= l synLhatred pixel clocjJ^^ioia^cxicrnal m^c*"" 30111 factor 

FC 311 also generates internal H sync and V^^ signals, 

which are used internally within the system. Specifically, the Thus, it should be noted from the above equation that 

"internal" V^^ signal is compared with the V^^ of the even though the external H^ MC frequency is unknown (and 

computer video signal 212 to determine and maintain ver- 40 may indeed never be known), the PLL 319 can automatically 

tical phasing. The "internal" V vnc signal is compared with recreate the original pixel clock of the computer video signal 

lne of the com P uter video signal 212 by the VFA 318 212 once the correct U total value is selected. In other words, 

to see if it is coincidental therewith. If it is, then it can be once the correct value for H totaJ is determined, it is possible 

concluded that the "internal" V^ is synchronized with the to correctly synchronize the external H^ ync with the internal 

v ^/ic °f lne computer video signal 212 (except for possible 45 H J ^ IC . As stated above, the determination of the correct H^, 

vertical phasing adjustments, as discussed below). The value is described below. 



sync 



3. Video Format Analyzer (VFA) 



or generate the original pixel clock from the H,^ signal of The VFAS 324 also includes a video format analyzer 318. 
the computer video signal 212 as described below. According to one embodiment, the VFA 318 is a pro- 
In order to create the internal signal, the FC 311 50 grammed array logic (PAL) containing a programmable 
divides the GCLK by a programmable integer value called counter, although it may constitute any similar logic device 
"fttota" arj d divides that quotient by a programmable value as known to those skilled in the art, such as an application- 
referred to as the "zoom factor" to create an internal specific integrated circuit or a field programmable gate array, 
signal, where H totaI is the number of group clock periods One embodiment of the VFA 318 is illustrated in FIG. 6. 
which occur in a horizontal scan line. This internal 55 A primary purpose of the VFA 318 is to analyze the 
signal is provided to the feedback input of the PLL 319. The computer video signal 212 to determine its video envelope 
pixel clock, generated by PLL 319 discussed below, can thus characteristics. In most cases, this analysis is performed by 
be programmably altered by modifying the zoom factor. The measuring the time between "events** of interest. "Events" of 
FC 311 creates the internal V^^. signal by dividing the interest involve the video timing parameters of the computer 
internal horizontal frequency (H^) by V^,, the latter also 60 video signal 212, and include: the leading edge of external 
being a programmable integer value equal to the total H^, the trailing edge of external H^, the leading edge of 
number of horizontal scan lines which occur in a single external V^, the trailing edge of external V,^, the leading 
frame. It should be noted that these internal and V^ edge of internal V^, the start of active video (both hori- 
signals are used for synchronization purposes and are sepa- zontal and vertical), and the end of active video (both 
rate and distinct from the and V^^. components, 65 horizontal and vertical). The reference time base for mea- 
respectively, of the computer video signal 212. For ease and surement of these parameters is GCLK (for measurement of 
clarity of understanding, therefore, the and V^^ events less than or equal to a single horizontal scan line) or 
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-syne v~* ~* ' than a According to one embodiment of the present invention, 

single scan line). For example, V foto/ (the number of hori- the video conference control application commands the 

zontal scan lines in a frame) is determined by counting in video driver to place a blue border of eight pixels around the 

counter 650, the number of "start of external H^^" events screen of monitor 220, prior to the measurement of vertical 

that occur between two successive "start of external V^, nc M 5 resolution by VFA 318, although such border is not needed 

events. until the VFA 318 begins the measurement of the active 

a. Sync Normalization by the VFA video region discussed in this paragraph. The video confer- 
In order for the VFA 318 to commence the various ence control application then sends instructions to the VFA 

measurements of the external H 0 , nc pulse, it should "know** 318 to commence measurement of the active video time of 

when such pulse is being asserted. The assertion of the 10 the computer video signal 212 once the blue border is 

external pulse is determined by a "sync normalization" detected. According to one embodiment of the present 

function in normalizer 652 which makes the determination invention, once measurement of the active video region has 

based upon the duty cycle of the pulse. Sync normalization completed, the video conference control application sends a 

is performed because the polarity of the H sy/IC and V^. of command over the bi-directional data and control path 215 

incoming computer video signal signals may vary depending is to extinguish the blue border. The video conference control 

on the host computer 210. For example, as phase -locked application controls the commencement and substantially 

loops such as PLL 319 operate on the edge of signals, PLL controls the termination of this operation (the multiplexing 

319 should be presented with an pulse of positive system may send interrupt commands to terminate the 

polarity. As seen with reference to FIG. 6, the normalizer operation in certain circumstances, otherwise the blue bor- 

652 accepts either polarity of H^^. and V sync and respec- 20 der will be extinguished automatically after a set time 

ttvely outputs an and V pulse of positive polarity. period). Alternatively, the video multiplexing system 213 of 

The normalizer 652 may be implemented in a number of the present invention may instead control the commence- 

ways. One implementation includes a counter which counts ment and termination of this operation. It should also be 

up when a sync signal is positive and down when a sync noted that although the "depth" of the border is eight pixels 

signal is negative; therefore, at the end of the counting 25 in one embodiment, such "depth** may instead constitute a 

period, the sign bit of the counter corresponds to the sign of different number of pixels and may constitute pixels of 

the sync pulse. This sign bit and the original sync pulse are another color (e.g., green or red, or a combination of the 

then provided as inputs to an exclusive -or (XOR) gate, so green, red and blue colors) in alternate embodiments, 

that the output is a positive polarity signal. Alternatively, the Additionally, it is to be appreciated that instead of the 

sync signal may be latched whenever active video is present, 30 automatic extinguishing of the blue border, if desired, the 

and since a sync pulse is not asserted during active video, the appearance of the blue border on the screen of monitor 220 

latched value will correspond to the sync polarity. It will be may be eliminated through known video technique such as 

appreciated that the manner by which the sync pulse is chroma-keying, 
normalized is not limited to these embodiments, and other 

methods of sync nonnalization may be used instead. 35 i. Vertical Measurement by the VFA 

b. Active Video Measurements by the VFA _ . , , . ~. , 
a • ^ c _, o . r . to ' „,ui„u tu* wn a vie Once the blue border is established, the comparator 654 of 
Again, one or the events or interest which the VFA 318 , , , . , . ' . ^ 

™*^,~o ic tuo ct,n a on A «f ,„t;,« x FIG. 6 signals when video is present. In the illustrated 

measures is the start and end oi active video time. In order .j i . t 1 1 *• t <• ■ i . . 

to measure the active video region of the computer video ^bodiment Ae foreshold for detection of video by the 

signal212, the VFA318 measures the vertical front and back 40 c ° m P arator 654 » °? 5 ™ lts . w !» ch » appropriate for most 

"porches" (respectively, V /re „„ and A as well vld f° s £ teas ^ ^ .y ,de ° ™Pf f™ 0.0 to 0.7 

aTthe horizorual front Ed porches ^spe^vely, vo1 *' ^ k start of act T ( V "' 1C ? ^asurement « 

H^,, and H^ t ^ of such signal. The VFA 318 P e ** med b * ™ n S m cou f; r 650 °™ ber * hon : 

. 7\r ^tZ ~ «-i *i u • zontal sync pulses that occur between the leading edge of 

SSTZTf&fin^ ^S&ttZ? fv ^.TT? « external V " and the horizontal scan line during which 

the tune from the leadmg edge of external V___ to the start 45 ., . Jf .-^ fIC , , ™ . - . r ■ ,^ 

of video (V„„„) and the time from the leading edge of vldeo 15 . de,ec ' cd ' ™ c k of . actl T c vlde ? 

external V to the end of video (V„J HoweverTto ensure measurem6nt » Performed by counting the number of hon- 

that the VFA 318 accurately measura V llarl and V tnJ , it is 2 ° Dtal s ,^ c ^ ^ bctwcCD ' h6 u lead ^ of 

required that the computer video signal contain known video f^f™ 1 V 3 " 1 ^.f^ hn f ^J^f™ Vlde0 15 

„„ k „ lt , fi „, j „ i; °n,', v i ■ i t . ,„ last detected. In a similar method, the VFA 318 can measure 

on both the nrst and last scan lines. This is accomplished by 50 ... . ... . . . , r 

u j r .i <• • the interval between events to determine vertical front porch, 

having a video conference control application or similar ,. .. . , . . , ^'"f""- u, - al " uul 

interface program in the computer 210 cause the video verUcal ^ P orch - ?" d verUc f 1 a f v = v.deo_aU measured 

driver (not shown) of monitor 220 to display a border m ™* of toninnUl ^nc_pubM (or horaonul scan lines) 

consisting of a number of pixels of a specified color at the " k ?°, wn ,0 f slull6d f m l , ^ 311 lo .« he 

boundary of the screen of monitor 220 for identifying the 55 resolutl ° n ° { thc «?««» of tDe com P uter video s lg nal 212 is 

active video region of the computer video signal. The *<™>"™y determined. 

requirement for me computer 210 to generate known video ~ Horizontal Measurements by the VFA 
content comes from the fact that the VFA 318 requires 

non-black video content at the borders to measure param- Horizontal measurements are made in a similar manner by 

eters such as front and back porch. However, in many 60 the VFA 318, the difference being that GCLK is used as the 

situations sufficient video content will always exist — such as unit of measurement. The start of horizontal active video 

when running Windows 95™ with the default blue desktop. measurement is performed by counting in counter 650 the 

In other situations, an enhancement to VFA 318 could be number of GCLK pulses which occur between the leading 

implemented to accumulate video timing information over a edge of the external H,^ pulse and the point when active 

period of time with the expectation that the computer 210 65 video is first detected. The end of horizontal active video 

would eventually display video information at the display measurement is performed by counting the number of 

borders. GCLK pules which occur between the leading edge of the 
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external pulse and the point when active video is last 
detected. Id addition to the above measurements, the VFA 
318 can measure the interval between events to determine 
horizontal front porch, horizontal back porch, horizontal 
active video — again all measured in units of GCLK, as 5 
known to those skilled in the art. All of these parameters will 
be used to determine horizontal resolution. 

However, as previously discussed, GCLK is a function of 
not only the external frequency but of the integer 

parameter H WB/ and the zoom factor (in the FC 311), and 10 
therefore it is important that H be accurately determined 
and specified to the FC 311 before any horizontal measure- 
ments are made. In order to ensure the accuracy of H^/, and 
thus determined horizontal resolution, this value is not 
directly determined by measurement, but instead is deter- 15 
mined by use of a lookup table in memory connected to 
controller 332. Determination of will now be dis- 

cussed. 

Measurement of and Video Synchronization by the 
Video Fornat Analyzer and Synchronizer (VFAS) 20 

FIG. 7 is a flowchart illustrating the steps associated with 
establishing video synchronization between the motion 
video signals and the computer video signal 212 according 
to one embodiment of the present invention. It is to be 
appreciated, however, that FIG. 7 illustrates only one 25 
embodiment, and that the order of the steps may be altered 
and/or steps removed. 

First, the VFA 318 measures the vertical resolution of the 
computer, step 710. As discussed above, the video confer- 
ence control application running on computer 210 causes the 30 
video driver in computer 210 to display a blue border on the 
screen of the monitor 220. Using this blue border, the VFA 
318 measures the vertical active video which is the vertical 
resolution of the computer video signal 212. 

After this measurement, the H total must be determined. 35 
Again, this value will be used to synthesize an internal 
and the synthesized pixel clock. In step 720, the controller 
332 uses the thus determined vertical resolution to look up 
the corresponding horizontal resolution according to Table I 
below which is stored in memory as discussed above, in 40 



order to determine H„ 



In alternate embodiments, other 



methods for determining horizontal resolution are used 
rather than a lookup table, such as calculating the horizontal 
resolution using equations well known in the art based on the 
vertical resolution. 

TABLE I 



Measured Vertical Resolution 


Expected Horizontal Resolution 


480 to 599 


640 


600 to 767 


800 


768 to 863 


3024 


864 to 1023 


.1152 


1024 to 1199 


3280 


1200 and above 


1600 



50 



55 



A value for H Wfo/ is then estimated by the following formula: 

\\ loto , (esl>1.2x (Expected Horizontal Resolution /4) 

Once the estimated H total has been calculated, the VFA 60 
318 measures the horizontal resolution of the computer 
video signal 212 by again measuring the interval between 
events of interest, step 730. The measured resolution is then 
compared to the expected resolution by VFA 318, step 740, 
and if the measured resolution is less than the expected 65 
resolution, H,^, is incremented by one, step 732, and step 
730 is repeated. This has the effect of increasing the fre- 



quency of the synthesized pixel clock. If the measured 
resolution is less than the expected resolution, is 
decremented by one, step 734, and step 730 is repeated. This 
has the effect of decreasing the frequency of the synthesized 
pixel clock. This process is repeated until the measured 
horizontal resolution is equal to the expected horizontal 
resolution. This ensures that the synthesized pixel clock has 
been correctly locked with the pixel clock of tie video signal 
212. 

In the embodiment shown in FIG. 7, the computer video 
signal 212 is then further analyzed to determine if the 
display of monitor 220 is interlaced or non-interlaced (step 
750). This is done in the VFA 318 by consecutive measure- 
ments of the time difference between the leading edge of 
and the leading edge of H^. In non-interlaced video, 
this time difference will be constant. In interlaced video this 
difference varies by half a scan line for each successive 
Vj^. If interlaced video is detected, the condition is flagged 
and operation may be aborted, step 755, if desired, or the 
process may continue and the remaining video timing 
parameters may be next measured and adjusted, step 760. 

Once deterrniaation is made as to whether the display is 
interlaced or not, the remaining video timing parameters are 
measured and adjusted accordingly in a manner similar to 
that discussed above, step 760. More specifically, the VFA 
318 measures the remaining horizontal and vertical param- 
eters and programs the FC 311 in accordance with these 
measurements. If desired, the video conference control 
application can be caused to extinguish the blue border. 

At this point, the motion video is now synchronized with 
the computer video signal 212 in all ways except for an 
expected phase shift in vertical sync. Thus, the next step, 
step 770, is to adjust the vertical phase of the motion video 
signal in order to match the vertical phase of the computer 
video signal 212. The VFA 318 accomplishes this by count- 
ing in counter 650 the H^. pulses between the leading edge 
of the internal pulse and the leading edge of the V J>MC 
pulse of the computer video signal 212. This count repre- 
sents the phase difference between the motion video signal 
and the computer video signal 212 in units of horizontal scan 
lines. This phase difference is corrected by increasing V^^ 
by the phase difference for a single frame. V tora/ is then 
immediately restored. The two video signals should now be 
completely synchronized. 

Another important feature of the VFA is that if vertical 
lock between the motion video and computer video signals 
is lost (that is, the internal V^^. is not synchronized with the 
external V^^), interrupt logic 656 of FIG. 6, under the 
control of control logic 658, will generate an interrupt. The 
VFA 318 can be programmed to prevent multiplexing of the 
motion video and computer video signals if synchronization 
is lost. In such a case, display of motion video on monitor 
220 of computer 210 will be blanked until vertical lock is 
re-established. Likewise, multiplexing will occur only when 
vertical lock is re-established. Alternatively, if desired, VFA 
318 can be programmed to multiplex the motion video and 
computer video signals irrespective of whether vertical lock 
is maintained. The latter is useful, for example, when trying 
to re-establish lock during video conferencing without los- 
ing the video conferencing channel altogether in the process, 
or if the display resolution is changed on the fly. 
Horizontal Screen Resolution Modification 

As discussed above, a separate frame buffer memory 
(VRAM 339 of FIG. 3) is used for the storage of video 
information for the motion video. In the illustrated 
embodiment, VRAM 339 is a 2 MB memory, with 1.5 MB 
being available for the display of motion video (the remain- 
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ing 0.5 MB being used for the storage of various control zoom control logic creates a "magnification" table, step 915. 

information and received images). Having 1.5 MB available The magnification table will be used to increase the hori- 

for the motion video display can result in storage space zontal screen resolution as discussed in more detail below, 

problems for large screen resolutions. The amount of However, if there is insufficient memory for the identified 

memory required to store an image can be identified by the 5 video signal resolution, then the zoom control logic creates 

following equation: a "zoom" table, step 917. The zoom table will be used to 

x decrease the horizontal screen resolution as discussed in 

memory -H^^^xV^^.^xcolor depth (in bytes) , , . „, . . , . . . 

more detail below. The zoom control logic then determines 

Thus, a 1024x768 screen resolution with 16 bits of color a new (modified) horizontal resolution, using either the 

requires 1024x768x2 bytes, which is 1,572,864 bytes, or 1.5 io magnification table from step 915 or the zoom table from 

MB. Similarly, a 1280x1024 screen resolution with 16 bits step 917, based on the available memory, step 920, and 

of color requires 1280x1024x2 bytes, which is 2,621,440 programs the PLL 319 based on the new horizontal 

bytes, or 2.5 MB. Thus, a screen resolution of 1280x1024 resolution, step 925. This programming of the PLL 319 

cannot be conventionally supported by a L5 MB memory alters the pixel width for the motion video to be different 

space. 15 than that used for the display of images from the computer 

The present invention resolves this problem by altering system 210. 

the width of the pixels (either enlarging the width or FIGS. 10 and 11 illustrate the pixel width modification 

reducing the width) in the motion video, a process referred according to one embodiment of the present invention. FIG. 

to as "zooming". By enlarging the pixel widths and corre- 10 illustrates a screen 1000 including an input scan line A 

spondingly reducing the horizontal resolution, the present 20 having a horizontal resolution of 1280 pixels (e.g., from 

invention is able to reduce the amount of memory space computer system 210), a concurrent scan line B having a 

necessary to store the image. Thus, by way of example, if the horizontal resolution of 768 pixels (e.g., zoomed motion 

screen resolution is 1280x1024 with 16 bits of color, the video), and a composite scan line C having two different 

present invention alters the widths of the pixels so that at horizontal resolutions (e.g., a combination of both the 

most 768 pixels per scan line are displayed. The effective 25 zoomed motion video and the video signal from computer 

screen resolution then becomes 768 (or less)xl024, which system 210). 

requires no more than 768x1024x2 bytes, which is 1,572, As illustrated, the pixels of the concurrent scan lines A, B, 

864 bytes, or 1.5 MB. and C "line up" with each other at both the start of the line 

According to one embodiment of the present invention, (the beginning of pixel 1 in each scan line) and at the end of 

the pixels are modified only in the horizontal dimension. 30 the line (the end of pixel 1280 in scan lines A and C and pixel 

That is, the effective horizontal screen resolution may be 768 in scan line B). However, each pixel in the scan line B 

modified, however the vertical screen resolution is not is wider than the pixels in the scan line A. By widening the 

altered. pixels in scan line B, fewer pixels are displayed per scan 

According to one embodiment, the present invention is line, and thus memory requirements for the display of the 

implemented in a combination of hardware and software. 35 scan line are reduced. 

The hardware components of overlaying motion video on an Additionally, the first four pixels and at least the last four 

analog signal are discussed above with reference to FIGS. pixels of scan line C (e.g., from the computer system 210) 

3-7. The software portion of the present invention is illus- are the same size as those in scan line A, while several of the 

trated in FIG. 8, which illustrates a storage medium 830 pixels in-between (e.g., from the local camera 222) are the 

including a zoom control portion 810 and a motion video 40 same size as those in scan line B. Scan line C illustrates a 

overlay control portion 820. The motion video overlay composite scan line of PC video and motion video, 

control portion 820 controls the setting of various param- The pixel widths of a scan line are modified by altering 

eters of the hardware components, as discussed above. The the synthesized pixel clock, as illustrated in FIG. 11. An 

zoom control portion 810 controls the identification of a H,^ signal 1110 is illustrated which indicates the beginning 

correct zoom factor and the setting of the hardware compo- 45 and ending of a horizontal scan line. The H^^ signal 1110 

nents based on this zoom factor, as discussed in more detail is provided as part of the video signal 212 from the computer 

below. The storage medium 830 is meant to represent a wide system 210. An active video line 1120 is illustrated which 

variety of machine-readable media in which the present indicates when active video is being displayed, either 

invention can be implemented, including conventional stor- images from the computer system 210 or the motion video, 

age devices (such as a floppy disk, random access memory, 50 Video is being displayed when the active video line 1120 is 

read only memory, Flash memory, etc.), as well as discrete "high" (that is, from time t 4 to time It should be noted 

hardware or firmware. In the illustrated embodiment, control that the active video line 1120 is included in FIG. 11 to assist 

portions 810 and 820 are implemented as a series of software in the explanation of the present invention; the active video 

routines executed by the video multiplexing system 213 of line 1120 is not an actual signal input to or generated by the 

FIG. 2. These software routines comprise a plurality or 55 video multiplexing system of the present invention. The time 

series of instructions to be executed by system controller between the end of H^, time t^, and the start of the active 

332 of FIG. 3. video, time t 4 , is referred to as the "back porch**. Similarly, 

FIG. 9 is a flowchart illustrating the steps followed in the time between the end of the active video, time to, and the 

modifying a horizontal screen resolution according to one start of the next H^^ time t 30 , is referred to as the "front 

embodiment of the present invention. As illustrated, the 60 porch". 

zoom control logic initially identifies the current video The display of pixels is dependent on a pixel clock. Anew 

signal resolution, step 905. This video signal resolution is pixel is displayed at each rising edge of the pixel clock, 

the resolution of the video signal 212 of FIG. 3. The zoom Thus, the number of pixel clock periods in the horizontal 

control logic then checks whether the identified video signal scan line corresponds to the number of pixels which are 

resolution is supported by the available memory (1.5 MB in 65 displayed on that scan line. A pixel clock 1130 is illustrated 

the illustrated embodiment), step 910. If there is sufficient as well as a synthesized pixel clock 1140. The pixel clock 

memory for the identified video signal resolution, then the 1130 is used for the display of pixels of images from video 
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signal 212, while the synthesized pixel clock 1140, gener- 
ated by PLL 319 of FIG. 3 discussed above, is used for the 
display of pixels of images from the motion video. As 
illustrated, a new pixel on the horizontal scan line is gen- 
erated at times t 4 , t 5 , and for images from analog RGB 
signal 212, while a new pixel on the horizontal scan line is 
generated at times t 3 , t 6 , and t 8 for the motion video. 

As discussed in more detail below, the video switch 314 
of FIG. 3 outputs video information from one of either video 
signal 212 or RGB signals from D/A converter 341 
(corresponding to the motion video). If the video informa- 
tion is output from video signal 212, then the pixels are 
displayed in accordance with pixel clock 1130, whereas if 
the video information is output from D/A converter 341, 
then the pixels are displayed in accordance with synthesized 
pixel clock 1140. 

In the illustrated embodiment, the video switch 31 zl can 
switch its output between the two source signals, video 
signal 212, and the RGB signals from D/A converter 341 at 
the beginning of every GCLIC Thus, it is preferred that the 
rising edge of the synthesized pixel clock 1140 line up 
closely with the rising edge of the pixel clock 1130 at the 
point where a switch from one source to another occurs. 

FIG. 12 is a flowchart illustrating the steps followed in 
identifying a proper horizontal resolution, based on avail- 
able memory, according to one embodiment of the present 
invention. The proper value for H toiaI from the computer 
system 210 (PC H^i) is first determined, step 1205, in 
accordance with the processes discussed above. The zoom 
control logic then determines a "modified" U total based upon 
pixels from the PC's point of view, step 1210. The modified 
U tatal value is initialized to be the same as H^ a/ , and is 
subsequently decremented by four for each iteration through 
step 1210. 

A zoom table entry is then calculated based on the 
modified H^,, step 1215. A zoom table entry includes at 
least four components: the modified H foto/ , the zoom factor, 
the zoomed horizontal resolution, and the alignment error. 
The modified H fe>w/ is the value determined in step 1210. The 
zoom factor is calculated by dividing U total by the modified 
U toIar The zoomed horizontal resolution (Z_HRES) is 
calculated by dividing the PC horizontal resolution by the 
zoom factor. The alignment error is calculated according to 
the following equation: 

error=zoom factorx(Z_HRES-(4.0x truncate ((Z_HRES/4.0>»- 
0-5)))) 

The zoom control logic then checks whether a sufficient 
number of zoom table entries have been calculated, step 
1220. According to one embodiment, the zoom control logic 
calculates 512 entries. However, alternate embodiments can 
generate a lesser or greater number of entries. Table II below 
illustrates a sample zoom table corresponding to a PC 
resolution of 1280x1024 and a PC H roto , of 1728. For ease 
of explanation, only a few entries have been included in 
Table II rather than 512. 

TABLE II 



Zoomed 
Horizontal 
Resolution 



Zoom 
Factor 



Alignment 
Error 



Modified 

H«xal 



1230 
1277.037 



1 

1.00232 



0.0 
1.0 



1728 
1724 



432 
431 



10 



50 



55 



60 



65 



16 



TABLE II -continued 



Zoomed 










Horizontal 


Zoom 


Alignment 


Modified 




Resolution 


Factor 


Error 




Howl 


1274.074 


1.004651 


-1.9 


1720 


430 


770.3704 


1.661538 


-2.7 


1040 


260 


767.4074 


1.667954 


-1.0 


1036 


259 


764.4444 


1.674419 


0.7 


1032 


258 


761.4815 


1.680934 


2.5 


1028 


257 


758.5185 


1.6875 


-2.5 


1024 


256 



20 " 



Once the zoom table is created, the zoom control logic 
uses the zoom table to identify a first zoomed resolution 
which fits into available memory, step 1225. This first 
zoomed resolution is the largest zoomed horizontal resolu- 
tion that can be used with the available memory. In the 
illustrated embodiment, with 16 bits of color (2 bytes), the 
first zoomed horizontal resolution (Z_HRES) is determined 
using the following calculation: 



25 



30 



35 



40 



45 



Available Memory *Z_HRESx Vertical Resolutionx2 

When the largest value of zoomed horizontal resolution 
(Z_HRES) in the table which satisfies the above calculation 
is identified, the zoom control logic evaluates a number of 
subsequent entries in the table, with lower zoomed horizon- 
tal resolutions, to identify a second zoomed resolution which 
provides a small alignment error, step 1230. This alignment 
error indicates the alignment difference of visible pixels 
between is internally generated video (from VRAM 339) 
and external generated video (from computer system 210). 
The zoom control logic searches for an alignment error 
within a predetermined range, (e.g., between -0.3 and 0.3). 
In the illustrated embodiment, the zoom control logic only 
looks at a few zoomed horizontal resolutions because each 
subsequent entry represents a loss of resolution. In the 
illustrated embodiment, the zoom control logic looks at a 
number of subsequent table entries, referred to as the "search 
range", identified by the following calculation: 

search length-alignment error at first Z_HRESx4 

The zoom control logic uses the largest zoomed horizontal 
resolution within the search range with an alignment error 
within the predetermined range as the second zoomed reso- 
lution. If no alignment error is within the predetermined 
range, then the zoom control uses the zoomed horizontal 
resolution within the search range with the smallest align- 
ment error as the second zoomed resolution. It is to be 
appreciated that the second zoomed horizontal resolution in 
step 1230 may be the same as the first zoomed horizontal 
resolution in step 1225. 

Generation of a magnification table (step 915 of FIG. 9) 
rather than a zoom table is done in an analogous manner, 
except that the magnification table entries are calculated by 
incrementing the modified H,^ value by four for each 
iteration through step 1210. Thus, zoom factors in the 
magnification table have a value between 0 and 1.0. The 
zoom control logic selects a first and a second zoomed 
horizontal resolution from the magnification table in the 
same maimer as it selects a first and second zoomed hori- 
zontal resolution from the zoom table. 

The zoom control logic then programs the frame control- 
ler 311 with this second zoomed horizontal resolution (from 
either the magnification table or the zoom table), step 1235. 
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According to one implementation, frame controller 311 is 
programmed with integer values, so the second zoomed 
horizontal resolution is thus rounded to the closest integer 
value. Thus, the frame controller will display the motion 
video with a horizontal resolution equal to this second 5 
zoomed horizontal resolution identified in step 1230. 

Thus, by way of example, referring to Table II with a color 
depth of 16 bits, the first zoomed horizontal resolution which 
fits into the available 1 .5 MB of memory is 767.4074. The 
zoom control logic then searches the next 4 entries and 10 
identifies the zoomed horizontal resolutions of 764.4444, 
which has an alignment error of 0.7, as the second zoomed 
resolution. The value of 764.4444 rounds to an integer value 
of 764, so 764 is thus programmed into the frame controller 
311. 15 

According to one embodiment of the present invention, 
the motion video which is overlaid onto the video signal 212 
is received as a Common Imaging Format (OF) image. The 
CIF standard uses an image dimension of 352x288, which is 
a common image resolution for current video conferencing 20 
applications. In a conventional video system, a scaling 
engine, such as co-processor 337 of FIG. 3 described above, 
is used to scale a lower resolution image to a higher 
resolution image. Such scaling typically involves the use of 
interpolation, in which the scaling engine calculates addi- 25 
tional pixel values for images based on the original image's 
pixel values. By way of example, a 352x288 resolution 
image can be scaled to a 1280x1024 resolution image by 
calculating 928 new pixel values for each horizontal line and 
738 new pixel values for each vertical column. Such scaling 30 
techniques are well known to those skilled in the art and thus 
will not be discussed further except as they pertain to the 
present invention. 

Thus, it is to be appreciated that the use of the present 
invention in a traditional video conferencing system does 35 
not affect the image quality of the motion video, despite the 
fact that the zoomed horizontal resolution of the CIF images 
may in fact be less than the resolution of the video signal 
212. At full screen size, the zoomed horizontal resolution 
(e.g., 768x1024), although less than the video signal 212 40 
resolution (e.g., 1280x1024) is still greater than the original 
image (e.g., 352x288), so no image quality has been lost. 

In the discussions above, the identification of the zoomed 
horizontal resolution to be used by the present invention is 
performed by generating a zoom or magnification table. 45 
However, it is to be appreciated that alternate embodiments 
need not generate such tables — by way of example, other 
calculations may be used, or the table entries for selected 
common resolutions may be pre-coded into a table (thereby 
eliminating an "on the fly" calculation). It is also to be 50 
appreciated that alternate embodiments can combine the 
zoom and magnification table into a single table rather than 
two separate tables. 

When zoom factors ranging between 0.0 and 1.0 are used, 
the synthesized pixel clock 1140 of FIG. 11 above has a 55 
shorter clock period than that of pixel clock 1130. By way 
of example, the image resolution of video signal 212 may be 
640x480. Using a 640x480 screen resolution for the motion 
video only requires 614,400 bytes (assuming 2 bytes of color 
per pixel), or approximately 0.6 MB. Thus, an additional 60 
approximately 0.9 MB of the VRAM 339 is available for 
display. Thus, the original screen resolution of 640x480 can 
be magnified to 1638x480 with a zoom factor of 0.39. 

Such an increase in screen magnification can increase the 
image quality of the motion video. For example, if a 65 
352x288 image were to be displayed in a window one- 
quarter the size of a screen with a resolution of 640x480, 
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then a window area with dimensions of 320x240 would be 
available for the image. However, since the image has a 
dimension of 352x288, then 32 pixels in the horizontal 
dimension and 48 pixels in the vertical dimension are "lost" 
by the scaling of co-processor 337. In other words, the 
scaling of co-processor 337 modifies the image to eliminate 
32 pixels in the horizontal dimension and 48 pixels in the 
vertical dimension. However, "magnifying" the motion 
video according to the present invention with a screen 
resolution of 1638x480, that same image being displayed on 
one-quarter of a screen would have a dimension of 819x240. 
Thus, although 48 pixels are still lost in the vertical 
dimension, no pixels are lost ill the horizontal dimension. 
Therefore, the quality of the image has been improved due 
to the co-processor 337 having to remove fewer pixels. 

Thus, particularly when used with a traditional video 
conferencing system, the present invention advantageously 
reduces the memory requirements for high screen resolu- 
tions without sacrificing image quality, and at the same time 
can increase image quality for lower screen resolutions. 

It should also be noted that although the above discus- 
sions describe the horizontal resolution modification as 
being preformed for an overlapping motion video, the 
present invention is not limited to such embodiments. In 
alternate embodiments, the present invention is imple- 
mented on "full screen" motion video. 
Smart Video Switch and Multiplexer Control Mask 

Once the motion video is processed appropriately, the 
aforementioned horizontal and vertical parameters of the 
display of the computer 210 determined, the zoom factor 
determined, and the motion video and computer video signal 
212 synchronized, the video multiplexing system 213 deter- 
mines whether the computer video signal 212 or the motion 
signal is to be displayed on the monitor 220. 

Referring again to FIG. 3, the smart video switch 314 is 
an analog switch used to source either the computer video 
signal 212 or the motion video output from the D/A con- 
verter 341 to the screen of computer monitor 220 through 
video amplifier 316. In operation, the smart video switch 
314 combines, or multiplexes, the computer video signal 
with the motion video output from the video D/A converter 
341, and then, based upon information generated by a 
multiplexer control mask (MCM) 334, selects the correct 
image source (computer 210 or motion video source) for 
display on the screen of monitor 220. According to one 
implementation, MCM 334 is a 16 k byte static random 
access memory (SRAM), while the smart video switch 314 
is a QS4A205 high speed analog multiplexer available from 
Quality Semiconductor, Inc. of Santa Clara, Calif. However, 
alternate embodiments can use similar commercially avail- 
able memories and/or multiplexers. 

In the present invention, the smart video switch 314 is 
capable of switching between images on a boundary of 
every four pixels, but may alternatively be made to switch 
on smaller boundaries (e.g., every two pixels or every pixel). 
While switching on smaller boundaries would increase the 
resolution of the border between the computer video signal 
212 from the host computer 210 and the video image to be 
displayed, it would also increase memory requirements of 
the MCM 334, and thus such increase in switch 314 switch- 
ing speed depends on the amount of memory available to the 
MCM 334. 

As each raster line of the combined video, motion video 
and computer video signal, is formed the system of the 
present invention determines which image source is to be 
displayed — the computer video signal 212 or the motion 
video signal output from video D/A converter 341. This 
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decision is controlled by the MCM 334. Under the control 
of controller 332, the MCM 334 controls the smart video 
switch 314 to output one of either the computer video signal 
212 from the computer 210 or motion video signal output 
from the video D/A converter 341. The 41 mask" serves as a 5 
template for drawing a background "window** into which the 
motion video signal from the local video source or remote 
video source will ultimately be displayed. Thus, the MCM 
334 is used to determine the display size and location of the 
window(s) which will display the video signal received from 10 
the remote and/or local video signal sources. 

More particularly, each bit in the MCM 334 controls the 
source of video for a pixel or multiple pixel block (e.g., a 
single bit in the MCM 334 controls a 4x4 pixel array in one 
implementation). The MCM 334 generates a continuous bit 15 
stream of pixel data having values of "0" or "1". Such values 
represent whether the switch 314 will source the computer 
video signal 212 from the computer 210 or whether it will 
source the motion video signal output from video D/A 
convert 341. According to one embodiment, a bit value of 20 
"0" will cause the smart video switch 314 to source the 
computer video signal 212 for four horizontally adjacent 
pixels of the screen, while a bit value of "1" will cause the 
smart video switch 314 to source the motion video signal 
output from the video D/A converter 341 for the four pixels 25 
for four scan lines. Again, resolution can be increased, if 
desired (e.g., in an alternate implementation a single bit in 
MCM 334 controls the source of video for a single pixel). 
The "windows" created by MCM 334 are much like other 
computer "windows" known in the art. Again, local video 30 
from the local video source 222 is ultimately displayed in 
one "window" (e.g., window 104 of FIG. 1), while video 
from the remote video source is ultimately displayed in 
another window (e.g., window 102 of FIG. 1). Information 
generated by the computer video signal 212 is displayed in 35 
another area, such as window 106 of FIG. 1. Additional 
"windows" may be displayed by the MCM 334. According 
to one embodiment, these windows are used to display 
graphics information from an application such as user inter- 
face program and which is used for controlling the system of 40 
the present invention. It is to be appreciated that the MCM 
334 may be programmed to display windows in rectangles 
or various other shapes. For example, if the granularity of 
the MCM is made to be very fine (e.g., a resolution of one 
or two pixels), it is possible to create a window which has 45 
an arbitrary shape. Thus, an outline view of the speaker 
could in fact be the "window". 

The content of the memory of MCM 334 may be con- 
trolled by the video conference control application or similar 
user interface program running in the computer 210. In the 50 
video conferencing embodiment, this application would 
control the size, position and stacking order of the near- and 
far-side windows. The user controls these factors in the same 
manner he or she would for any computer window. For 
example, if the user resizes or moves a window containing 55 
the near- or far-side motion video, this information is 
transmitted from the computer system 210 to the multiplexer 
system of the present invention through controller 332 and 
bus arbitration and control logic 345, the system controller 
332 adjusting the contents of the MCM 334 memory accord- 60 
ingly. 

Although the multiplexing system of the present inven- 
tion has been discussed in reference to a video conferencing 
environment, it will be appreciated that such multiplexing 
system has a broader range of uses, and may be used in 65 
several applications, such as a communications broadcast 
environment, a multimedia environment, a DVD-recorder 
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and/or playback application, other computer and video 
applications, etc. Additionally, such multiplexing system 
may be used in a video test instrument or in a video format 
converter. Furthermore, although the source of the computer 
video signal 212 is described above as a computer outputting 
VGA^style RGB video with separate and V^. pulses, 
the present invention is not so limited, and may be extended 
to all versions of RGB video, including video with compos- 
ite H^^. and video with sync on green, as well as RS 
170 compliant video and its derivatives. In addition, 
although control of the motion video frame controller 311 is 
effected using software commands, it may alternatively be 
effected using a hardware implementation. 

According to an alternate embodiment of the present 
invention, multiplexing system 213 provides memory man- 
agement based on window sizes and positions. In this 
alternate embodiment, rather than having the entire screen 
which is zoomed by the present invention, a region (e.g., a 
single rectangular region) of the display is defined which 
encompasses all of the windows containing motion video. 
The resolution of this region is used to compute the proper 
zoom factor rather than the resolution of the entire screen. 
This region is then considered to be the motion video and is 
zoomed as discussed above. According to one implementa- 
tion this region is dynamically defined and is modified each 
time a window within the region is closed or re-sized, or a 
new window on the display device is opened (e.g., either a 
newly created window or a previously minimized window 
being re -stored to its previous size). 

FIG. 13 illustrates a display according to One such 
alternate embodiment. The display 1300 includes two win- 
dows 1301 and 1302. A region 1305 is defined by the zoom 
control logic which includes both of the windows 1301 and 
1302. As illustrated, although the computer system is using 
a screen resolution of 1280x1024, only a 900x600 region 
includes windows being displayed. Thus, the zoom control 
logic calculates a zoom factor based on a computer screen 
resolution of 900x600 rather than 1280x1024. Thus, a zoom 
factor of less than 1 could be used even though the computer 
resolution suggests a zoom factor greater than 1 due to the 
implied requirement of 2.5 MB (1280x1024x2 bytes). 

Furthermore, although the multiplexing system of the 
present invention has been discussed in reference to receiv- 
ing motion video from a camera, any of a wide variety of 
devices can be the source of such motion video. Examples 
of such sources include a video cassette player, laser disk 
player, television, digital versatile disk (DVD) player, etc. 

In addition, although the multiplexing system of the 
present invention has been discussed in reference to receiv- 
ing a motion video signal, the present invention is not 
limited to such embodiments. In alternate embodiments 
other types of signals are received, such as "static** or "still" 
images (e.g., one or more frames of the same image), from 
any of a wide variety of conventional sources. By way of 
example, VRAM 339 of FIG. 3 may repeatedly provide the 
same data to D/A converter 341 for multiple frames of 
computer video signal 212. By way of another example, the 
same image may repeatedly be input to VRAM 339. It is to 
be appreciated that the zooming of the present invention is 
not dependent on the source of the video signal or the video 
content (e.g., motion video or otherwise). 

Additionally, although the zooming of the present inven- 
tion has been discussed in reference to a video multiplexing 
system, the present invention is not limited to such embodi- 
ments. In alternate embodiments, the zoomed motion video 
encompasses the entire display of monitor 220 rather than 
being an overlay. Thus, it is to be appreciated that in sucb 
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alternate embodiments much of the hardware of multiplex- first candidate resolution and a second candidate resolution, 

ing system 213 for synchronizing the motion video to the comparing each candidate resolution to the available frame 

computer video signal is not necessary, as there are not two buffer memory, selecting a candidate resolution based on the 

different signals to be synchronized. comparison. 

Furthermore, in the discussions above, the multiplexing 5 13. The method of claim 12, wherein the step of creating 

system 213 is described as being separate from the computer the second screen resolution further comprises generating 

system 210. It is to be appreciated that, in alternate further candidate resolutions by incrementing the resolution 

embodiments, multiplexing system 213 can be incorporated by a selected amount. 

into computer system 210. 14. The method of claim 12, wherein the step of selecting 
Thus, a method and apparatus for modifying screen 1Q a candidate further comprises selecting the largest resolution 
resolution based on available memory has been described. that will fit into the available frame buffer memory. 
Whereas many alterations and modifications of the present 15. The method of claim 1, wherein the step of creating 
invention will be comprehended by a person skilled in the art the second screen resolution further comprises generating a 
after having read the foregoing description, it is to be first candidate resolution and a second candidate resolution, 
understood that the particular embodiments shown and determining the alignment error between each candidate 
described by way of illustration are in no way intended to be 15 resolution and the first resolution, comparing each candidate 
considered limiting. References to details of particular resolution to the available frame buffer memory, and select- 
embodiments are not intended to limit the scope of the m g a candidate resolution with the largest resolution that 
claims. ^1 & mt0 me available frame buffer memory from among 
What is claimed is: mose cand i dale resolutions that have an alignment error less 

1. A method comprising the steps of: 20 man a desired threshold. 

identifying a first screen resolution for a display, the first , 16 , A machinwcadtble medium having stored thereon a 

screen resolution corresponding to a resolution setting P luraht y °/ jettons, designed to be executed by a 

for the display- processor, for implementing a function for identifying a first 

receiving a video' signal to be displayed on the display screeD res ° lutioD for a display, the first screen resolution 

having a video signal resolution; and 25 corresponding to a resolution setting for the display, for 

creating a second screen resolution for displaying the receivia S a / deo si S Dal t0 be tepbytd on the display 

received video signal on the display based on the video havm g a Vlde0 sl S nal ^solution, and for creating a second 

signal resolution, the first screen resolution and a screen resolution for displaying the received video signal on 

comparison of the second screen resolution to the the display based on the video signal resolution, the first 

amount of memory available in a frame buffer for the 30 screen resolution and a comparison of the second screen 

display, to create the largest second screen resolution resolution to the amount of memory available in a frame 

that can be used with the memory available. buffer for the display, to create the largest second screen 

2. The method of claim 1, wherein the step of creating resolution that can be used with the memory available, 
comprises the step of altering a pixel width for each of a 17. The machine-readable medium of claim 16, wherein 
plurality of pixels of the received video signal. 35 the plurality of instructions for implementing a function for 

3. The method of claim 2, wherein the step of creating creating comprises a plurality of instructions for implement- 
further comprises the steps of: ing a function for altering a pixel width for each of a 

identifying a plurality of screen resolutions which can be plurality of pixels of the received video signal, 

used as the second screen resolution; and 18. The machine -readable medium of claim 17, wherein 

selecting as the second screen resolution the one of the 40 the plurality of instructions for implementing a function for 

plurality of screen resolutions which provides a small- creating are further for implementing a function for identi- 

est alignment error with the first screen resolution. fying a plurality of screen resolutions which can be used as 

4. The method of claim 2, wherein the step of altering a the second screen resolution, and for selecting as the second 
pixel width comprises the step of altering a pixel clock screen resolution the one of the plurality of screen resolu- 
period corresponding to the second screen resolution. 45 tions which provides a smallest alignment error with the first 

5. The method of claim 4, wherein the step of altering the screen resolution. 

pixel clock period comprises the step of increasing the pixel 19. The machine-readable medium of claim 17, wherein 

clock period. the plurality of instructions for implementing a function for 

6. The method of claim 2, wherein the plurality of pixels altering a pixel width comprises a plurality of instructions 
comprises every pixel to be displayed as part of the image. 50 for implementing a function for altering a pixel clock period 

7. The method of claim 1, wherein displaying the received corresponding to the second screen resolution. 

video signal on the display comprises displaying the 20. The machine-readable medium of claim 19, wherein 

received video signal on the entire display. the plurality of instructions for implementing a function for 

8. The method of claim 1, further comprising the step of altering the pixel clock period comprises plurality of instmc- 
overlaying an image having the second screen resolution on 55 tions for implementing a function for increasing the pixel 
an image having the first screen resolution. clock period. 

9. The method of claim 1, wherein the received video 21. The machine-readable medium of claim 17, wherein 
signal includes one or more windows which provide video the plurality of pixels comprises every pixel to be displayed 
information from one or more video conferencing cameras. as part of the image. 

10. The method of claim 1, wherein a horizontal resolu- 60 22. The machine-readable medium of claim 16, wherein 
tion of the second screen resolution is less than a horizontal displaying the received video signal on the display com- 
resolution of the first screen resolution. prises displaying the received video signal on the entire 

U. The method of claim 1, wherein a horizontal resolu- display, 

tion of the second screen resolution is greater than a hori- 23. The machine-readable medium of claim 16, wherein 

zontal resolution of the first screen resolution. 65 the plurality of instructions are further for implementing a 

12. The method of claim 1, wherein the step of creating function for overlaying an image having the second screen 

the second screen resolution further comprises generating a resolution on an image having the first screen resolution. 
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24. The machine-readable medium of claim 16, wherein 
the received video signal includes one or more windows 
which provide video information from one or more video 
conferencing cameras. 

25. The machine-readable medium of claim 16, wherein 5 
a horizontal resolution of the second screen resolution is less 
than a horizontal resolution of the first screen resolution. 

26. The machine-readable medium of claim 16, wherein 
a horizontal resolution of the second screen resolution is 
greater than a horizontal resolution of the first screen reso- 1Q 
lution. 

27. The machine-readable medium of claim 16, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 
ing a function for generating a first candidate resolution and 

a second candidate resolution, comparing each candidate 15 
resolution to the available frame buffer memory, selecting a 
candidate resolution based on the comparison. 

28. The machine-readable medium of claim 27, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 20 
ing a function for generating further candidate resolutions by 
incrementing the resolution by a selected amount. 

29. The machine-readable medium of claim 27, wherein 
the plurality of instructions for implementing a function for 
selecting a candidate comprises a plurality of instructions for 2 $ 
implementing a function for selecting the largest resolution 
that will fit into the available frame buffer memory. 

30. The machine-readable medium of claim 16, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 3Q 
ing a function for generating a first candidate resolution and 

a second candidate resolution, determining the alignment 
error between each candidate resolution and the first 
resolution, comparing each candidate resolution to the avail- 
able frame buffer memory, and selecting a candidate reso- 35 
lution with the largest resolution that will fit into the 
available frame buffer memory from among those candidate 
resolutions that have an alignment error less than a desired 
threshold. 

31. An apparatus comprising: 

a video format analyzer and synchronizer for identifying 
a first screen resolution for a display, the first screen 
resolution corresponding to a resolution setting for the 
display and for receiving a video signal to be displayed 
on the display having a video signal resolution; and 

a zoom control logic, coupled to the video foimat analyzer 45 
and synchronizer, for creating a second screen resolu- 
tion for displaying the received video signal on at least 
a portion of the display based on the video signal 
resolution, the first screen resolution and a comparison 
of the second screen resolution to the amount of 50 
memory available in a frame buffer for the display, to 
create the largest second screen resolution that can be 
used with the memory available. 

32. The apparatus of claim 31, wherein the apparatus 
comprises a video multiplexing system. 55 

33. The apparatus of claim 31, wherein the zoom control 
logic is also for altering a pixel width for each of a plurality 
of pixels of the received video signal. 

34. The apparatus of claim 33, wherein the zoom control 
logic is also for identifying a plurality of screen resolutions 60 
which can be used as the second screen resolution, and for 
selecting as the second screen resolution the one of the 
plurality of screen resolutions which provides a smallest 
alignment error with the first screen resolution. 

35. The apparatus of claim 33, wherein the zoom control 65 
logic is also for altering a pixel clock period corresponding 

to the second screen resolution. 
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36. The apparatus of claim 33, wherein the zoom control 
logic is also for increasing the pixel clock period. 

37. The apparatus of claim 33, wherein the plurality of 
pixels comprises every pixel to be displayed as part of the 
image. 

38. The apparatus of claim 31, wherein the received video 
signal is displayed on the entire display. 

39. The apparatus of claim 31, further comprising a video 
switch, coupled to both the zoom control logic and the video 
format analyzer and synchronizer, for overlaying an image 
having the second screen resolution on an image having the 
first screen resolution. 

40. The apparatus of claim 31, wherein the received video 
signal includes one or more windows which provide video 
information from one or more video conferencing cameras. 

41. The apparatus of claim 31, wherein a horizontal 
resolution of the second screen resolution is less than a 
horizontal resolution of the first screen resolution. 

42. The apparatus of claim 31, wherein a horizontal 
resolution of the second screen resolution is greater than a 
horizontal resolution of the first screen resolution. 

43. The apparatus of claim 31, wherein the zoom control 
logic is also for generating a first candidate resolution and a 
second candidate resolution, comparing each candidate reso- 
lution to the available frame buffer memory, and selecting a 
candidate resolution based on the comparison. 

44. The apparatus of claim 43, wherein the zoom control 
logic is also for generating further candidate resolutions by 
incrementing the resolution by a selected amount. 

45. The apparatus of claim 43, wherein the zoom control 
logic is also for selecting the largest resolution that will fit 
into the available frame buffer memory. 

46. The apparatus of claim 31, wherein the zoom control 
logic is also for generating a first candidate resolution and a 
second candidate resolution, determining the alignment 
error between each candidate resolution and the first 
resolution, comparing each candidate resolution to the avail- 
able frame buffer memory, and selecting a candidate reso- 
lution with the largest resolution that will fit into the 
available frame buffer memory from among those candidate 
resolutions that have an alignment error less than a desired 
threshold. 

47. An apparatus comprising: 

means for identifying a first screen resolution for a 
display; 

means for receiving a video signal to be displayed on the 
display having a video signal resolution; and 

means, coupled to the means for idenitif ying, a first screen 
resolution, for creating a second screen resolution for 
displaying the received video signal on the display 
based on the video signal resolution, the first screen 
resolution and a comparison of the second screen 
resolution to the amount of memory available in a 
frame buffer for the display, to create the largest second 
screen resolution that can be used with the memory 
available. 

48. The apparatus of claim 47, wherein the means for 
creating is also for altering a pixel width for each of a 
plurality of pixels of the received video signal. 

49. The apparatus of claim 48, wherein the means for 
creating is also for identifying a plurality of screen resolu- 
tions which can be used as the second screen resolution, and 
for selecting as the second screen resolution the one of the 
plurality of screen resolutions which provides a smallest 
alignment error with the first screen resolution. 

50. The apparatus of claim 48, wherein the means for 
creating is also for altering a pixel clock period correspond- 
ing to the second screen resolution. 
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51. The apparatus of claim 50, wherein the means for creating, for overlaying an image having the second screen 
creating is also for increasing the pixel clock period. resolution on an image having the first screen resolution. 

52. The apparatus of claim 47, further comprising means, 

coupled to both the means for identifying and the means for ***** 
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